Conversation
📝 WalkthroughWalkthroughPropagate a per-question Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
backend/app/crud/evaluations/langfuse.py (1)
254-296: Fixquestion_idtype hint (int vs str).
question_idis assigned as an integer, but the helper signature declaresstr, which will trip type checkers and contradicts tests.🐛 Suggested fix
- def upload_item(item: dict[str, str], duplicate_num: int, question_id: str) -> bool: + def upload_item(item: dict[str, str], duplicate_num: int, question_id: int) -> bool:
🧹 Nitpick comments (3)
backend/app/tests/crud/evaluations/test_processing.py (1)
70-98: Add a type annotation to the new test method signature.Per the project requirement, parameters should be typed; this applies to test methods too.
As per coding guidelines, please add type hints to all function parameters.♻️ Suggested update
-from typing import Any +from typing import Any, Self @@ - def test_parse_evaluation_output_without_question_id(self) -> None: + def test_parse_evaluation_output_without_question_id(self: Self) -> None:backend/app/tests/crud/evaluations/test_langfuse.py (1)
499-571: Type annotatevalid_itemsin the new tests.The new test methods introduce untyped parameters.
As per coding guidelines, please add type hints to all function parameters.♻️ Suggested update
- def test_upload_dataset_to_langfuse_question_id_in_metadata(self, valid_items): + def test_upload_dataset_to_langfuse_question_id_in_metadata( + self, valid_items: list[dict[str, str]] + ) -> None: @@ - def test_upload_dataset_to_langfuse_same_question_id_for_duplicates( - self, valid_items - ): + def test_upload_dataset_to_langfuse_same_question_id_for_duplicates( + self, valid_items: list[dict[str, str]] + ) -> None:backend/app/crud/evaluations/langfuse.py (1)
424-447: Keepquestion_idtype consistent in fetched trace data.Defaulting to
""introduces a str/int mix; preferNonewhen absent to align with upstream.♻️ Suggested update
- "question_id": "", + "question_id": None, @@ - trace_data["question_id"] = trace.metadata.get("question_id", "") + trace_data["question_id"] = trace.metadata.get("question_id")
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
backend/app/crud/evaluations/langfuse.py (1)
256-297: Addquestion_idfield toTraceDataTypedDict and normalize type toint.The
question_idfield is missing from theTraceDataTypedDict definition inbackend/app/crud/evaluations/score.py(lines 20–27), yet the code attempts to set it at lines 421–427 and 449. Additionally, the function parameter is annotated asstr(line 256) but assigned asintfrom enumerate (line 287), then defaulted to""string (line 426) when reading.Fix:
- Add
question_id: intto theTraceDataTypedDict (score.py)- Change the parameter annotation from
question_id: strtoquestion_id: int(langfuse.py, line 256)- Initialize
trace_data["question_id"] = 0instead of""(langfuse.py, line 426)- Convert the metadata value to int on read:
trace_data["question_id"] = int(trace.metadata.get("question_id", 0)) if trace.metadata.get("question_id") else 0(langfuse.py, line 449)
2 participants
Summary
Target issue is #544
Checklist
Before submitting a pull request, please ensure that you mark these task.
fastapi run --reload app/main.pyordocker compose upin the repository root and test.Notes
New Features
Tests